Information Structures for Feedback Capacity of Channels with Memory and Transmission Cost: Stochastic Optimal Control&Variational Equalities-Part I

The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions { PBi|Bi−1,Ai : i= 0,1, . . . ,n } and { PBi|Bi−1 i−M ,Ai : i= 0,1, . . . ,n } , where M is the memory of the channel, Bn 4 = {B j : j = 0,1, . . . ,n} are the channel outputs and An 4 = {A j : j = 0,1, . . . ,n} are the channel inputs. The characterizations of FTFI capacity, are obtained by first identifying the information structures of the optimal channel input conditional distributions P[0,n] 4 = { PAi|Ai−1,Bi−1 : i= 1, . . . ,n } , which maximize directed information CFB An→Bn 4 = sup P[0,n] I(An→ Bn), I(An→ Bn) 4 = n ∑ i=0 I(A;Bi|B). The main theorem states, for any channel with memory M, the optimal channel input conditional distributions occur in the subset ◦ P [0,n] 4 = { PAi|Bi−1 i−M : i = 1, . . . ,n } , and the characterization of FTFI capacity is given by C An→Bn 4 = sup ◦ P [0,n] n ∑ i=0 I(Ai;Bi|B i−M). Similar conclusions are derived for problems with transmission cost constraints of the form 1 n+1 E { ∑i=0 γi(Ai,T iBn−1) } ≤ κ,κ > 0, where { γi(Ai,T iBn−1) : i = 0,1, . . . ,n } is any class of multi-letter functions such that T iBn−1 = {Bi−1,Bi−2, . . . ,Bi−K} or T iBn−1 = {Bi−1}, for i = 0, . . . ,n and K a nonnegative integer. The methodology utilizes stochastic optimal control theory, to identify the control process, the controlled process, and a variational equality of directed information, to derive upper bounds on I(An→ Bn), which are achievable over specific subsets of channel input conditional distributions P[0,n]. For channels with limited memory, this implies the transition probabilities of the channel output process are also of limited memory. For any of the above classes of channel distributions and transmission cost functions, a direct analogy between the characterizations of FTFI capacity and Shannon’s capacity formulae of Memoryless Channels is identified.